Different Modes Of Data Collection Research Articles

Which Scale Direction is More Difficult for Respondents to Use? An Eye-tracking Study

Scale direction is found to affect response distributions by yielding more selections of scale points closer to the beginning of the scale. Although this phenomenon has been empirically demonstrated in different modes of data collection for respondents with different demographic characteristics, it is not clear which scale direction is cognitively difficult for respondents to use. Eye-tracking is used to address this research question because it provides a direct window into how respondents process survey questions. I compared dilation and fixation measures across scale direction and found that the satisfaction scale and the frequency scale running from “Never” to “Very Often” had more and longer fixations. In addition, the dissatisfaction options incurred larger dilations, more peak dilations, and more and longer fixations than the other parts of the scale. Findings of this paper shed light on how respondents process response scales and have practical implications for questionnaire design. Furthermore, findings of this paper demonstrate the utility and potential of using eye-tracking in general and dilation measures in particular to understand cognitive burden of answering survey questions.

Analyzing the Effect of Regional Modality in Polling Surveys: A Case Study of the 2020 U.S. Presidential Election Results in Florida

Pre-election polls have been conducted using different modes of data collection with varying degrees of accuracy. As media’s appetite for horserace polling increased, the demand for all types of data collection increased, raising the price of achieving a representative sample. Mixing modes of data collection have been popular due to the increased cost of polling using traditional methods such as face-to-face interviews, mail-in surveys, or live operator phone surveys. Due to technological advances in self-administered surveys, other forms of data collection provide more cost-effective solutions. Florida Atlantic University’s Business and Economics Polling Initiative conducted political polls throughout the fall of 2020 in anticipation of the general election for President of the United States. This unique dataset from polls conducted in September and October of 2020 showed interactive voice response (IVR) landline polling to be the most accurate statewide and across most regions of Florida. The survey data collected via opt-in online panels and via mobile phone text messaging exhibited results outside of the margin of error for certain regions. However, given the trend of declining landline phones and greater instances of exclusive mobile phone usage, polling by way of online and mobile phone surveys is becoming increasingly important.

In the Mode. . .Text-to-Web Survey Data Collection: An Exploratory Study in Preelection Polling of the U.S. Presidential Election

As our society rapidly employs new forms of communication, new modes of data collection are challenging the best practices developed over years of polling. Preelection polling must simultaneously evolve, as new modes have emerged in the past few decades, including computer-mediated communication, mobile texting, and the use of touch tone keypads to communicate information. A tension exists between traditional and novel means of interpersonal communication, and researchers are struggling to determine which traditional methods of data collection still have a place in the modern industry. This study examined three relatively new modes of preelection poll data collection, online, mobile, and IVR (interactive voice recognition) to determine what relationships exist, if any, between the mode of data collection and the composition of a sample across eight demographic variables: age, education, gender, political affiliation, race, region, 2016 Vote History, and 2020 Vote Intention. Twenty-six preelection polls were used in the study, with each poll ranging in collection dates between August 30 and October 31, 2020. The total combined sample size for this study is n = 19,886; 49% were IVR respondents ( n = 9,795), 25% was collected from online panels ( n = 5,039), and 25% was collected from short message service (SMS)-to-web respondents ( n = 5,052). A χ2 (chi-square) test for association was conducted using a significance level of p < .05 and a 95% confidence interval (CI) and found a significant difference between each mode of data collection across the eight aforementioned variables. A significant difference between political party affiliation/registration and mode of data collection was attributed to the educational attainment of individuals participating in each preelection polls based on the mode of data collection. This study suggests that underlying variables within the sample composition of different modes of data collection can have an impact on the accuracy of preelection polls.

Concurrent Mixed Modes: Response Quality, Speed, and Cost

This article presents the results of a general population study that used three different modes of data collection administered sequentially. The study began with a letter that contained the link to an online survey. Those who did not respond were interviewed by phone or face-to-face. The article focuses on the cost of the study related to response quality, and hypothesizes that the sequential use of various modes, first using the most economical ones, yields significant cost savings that do not affect the quality of the information obtained. The results indicate that the online survey was nine times cheaper than interviewer-administered modes of data collection. The face-to-face survey was the one with the highest rate of item nonresponse, whereas the telephone survey had the lowest item nonresponse rate.

Survey mode and data quality: Careless responding across three modes in cross-cultural contexts

Much psychological research depends on participants’ diligence in filling out materials such as surveys. However, not all participants are motivated to respond attentively, which leads to unintended issues with data quality, known as careless responding. Our question is: how do different modes of data collection—paper/pencil, computer/web-based, and smartphone—affect participants’ diligence vs. “careless responding” tendencies and, thus, data quality? Results from prior studies suggest that different data collection modes produce a comparable prevalence of careless responding tendencies. However, as technology develops and data are collected with increasingly diversified populations, this question needs to be readdressed and taken further. The present research examined the effect of survey mode on careless responding in a repeated-measures design with data from three different samples. First, in a sample of working adults from China, we found that participants were slightly more careless when completing computer/web-based survey materials than in paper/pencil mode. Next, in a German student sample, participants were slightly more careless when completing the paper/pencil mode compared to the smartphone mode. Finally, in a sample of Chinese-speaking students, we found no difference between modes. Overall, in a meta-analysis of the findings, we found minimal difference between modes across cultures. Theoretical and practical implications are discussed.

Intra-Individual Variability in Self-Reported Use of Non-Steroidal Anti-Inflammatory and Analgesic Medicines Depending on Mode of Data Collection - Observations from the Population Study of Women in Gothenburg, Sweden.

BackgroundSelf-report by questionnaire is one of the main methods to collect data on drug utilization. There are several modes of data collection by questionnaire, differing in the way of delivering the questionnaire to respondents and in the administration of the questions, both influencing the recall and participation rates. The aim of this study was to compare different modes of data collection for self-reported use of non-steroidal anti-inflammatory (NSAIDs) and analgesic medicines.MethodsData on 573 women (38 or 50 years) were retrieved from the Population Study of Women in Gothenburg. Data on medicine use were collected using two different modes: (1) a self-administered questionnaire with closed-ended medicine-specific questions; and (2) an interviewer-administered questionnaire with open-ended questions. Cohen’s kappa statistics were applied to assess the agreement of the two modes.ResultsThe proportion of participants that reported use of NSAIDs and analgesics was higher with the self-administered questionnaire compared with the interviewer-administered questionnaire (69.3% vs 58.5%, p <0.001). The overall agreement between the two modes of data collection was fair (Ⱪ=0.27), ranging from none for antimigraine preparations to fair (Ⱪ=0.36) for NSAIDs. A higher proportion of the participants aged 38 years reported use of NSAIDs and analgesics compared with the 50-year olds. In the regression model using data from the self-administered questionnaire, all four categories of bodily pain were significant predictors for use of NSAIDs and analgesics. The most severe reported bodily pain was the only significant predictor in the model using data from the interviewer-administered questionnaire.ConclusionThis study showed that use of a self-administered questionnaire with closed-ended medicine-specific questions identified more users of NSAIDs and analgesic medicines compared with an interviewer-administered questionnaire with open-ended questions. Reported use according to the self-administered questionnaire was also more strongly associated with experienced pain.

Photoelectron diffraction: Early demonstrations and alternative modes

A brief review of the early demonstrations of the utility of photoelectron diffraction to determine the structure of surfaces, contrasting the two fundamentally different approaches of high-energy forward scattering and low-energy backscattering, and the alternative angle-scan and energy-scan different modes of data collection in backscattering experiments is presented. The development and applications of the high-energy angle-scan x-ray photoelectron diffraction version of the technique by Fadley and co-workers is contrasted with those of low-energy backscattering photoelectron diffraction, with particular emphasis on studies of molecular adsorption using the energy-scan mode (PhD).

Are online job quality quizzes of any value? Selecting questions, maximising quiz completions and estimating biases

PurposeThe purpose of this paper is to compare two ways of collecting job quality data in Britain using a common set of questions. One way is through a short quiz taken by a self-selected sample and completed by clicking on a web link www.howgoodismyjob.com. The other way is via an invitation to take part in a long-running survey of working life – the Skills and Employment Survey. The survey takes much longer to complete, is carried out face-to-face and is based on random probability principles.Design/methodology/approachTo be content-comparable, the quiz uses tried and tested questions contained in recent waves of the Skills and Employment Survey. Each survey comprises a nationally representative sample of workers in Britain aged 20–65 years. However, the quiz is based on uncontrolled convenience sampling prompted, in large part, by a Facebook advertising campaign, whereas survey participants are randomly selected. In this paper, the authors compare the profile of respondents and their responses to these two different modes of data collection and therefore shine a light on any biases in the samples and differences in the results respondents report.FindingsThe paper shows that while the number taking in the quiz is impressive, participation in the quiz – unlike the survey – is heavily skewed. Weighting can be used to correct some of these sample selection biases. But, even then, the picture painted by the quiz and survey data varies with the quiz under-reporting the intrinsic quality of jobs, while over-reporting on the extrinsic rewards. This suggests that how job quality data are collected can have a strong influence on the results produced.Research limitations/implicationsThe findings suggest that a number of biases are in operation, both in terms of those who take part and the answers they give. This makes comparison between data collected using radically different methods, at best, inadvisable and, at worst, misleading. Nevertheless, quizzes are a good way of engaging large numbers of people in public debates, gathering additional data, extending the reach of academic work and prompting action to improve working life. However, the limitation of this study is that it does not offer a true experiment of different ways of collecting the same data. The quiz and survey were, for example, not carried out at the same time, but were some 14 months apart.Practical implicationsOver 50,000 people took part in the quizzes reported in the paper and almost 1,300 investigated joining a trade union as a result. The reach of the quiz far exceeds the 3,306 people who took part in the Skills and Employment Survey 2017.Originality/valueThis paper focusses on how job quality data are collected and the consequences this has for the validity of the data gathered. This is a unique contribution to international debates about the measurement and monitoring of trends in job quality.

Collecting Data to Facilitate Change

Data collection and analysis are central to all quality improvement initiatives. However, there is often a lack of understanding about the different ways data can be collected and why one mode of data collection should be selected over another. In this article, a framework is provided to help quality improvement teams identify and define the data to be collected. This is followed by a discussion of the different modes of data collection. Examples are provided to help illustrate each mode of data collection. Finally, the advantages and disadvantages of each mode are described.

Survey Mode and Data Quality: A Cross-Cultural Comparison of Careless Responding Across Three Modes

Much psychological research depends on participants’ diligence in filling out materials such as tests or surveys. However, not all participants are motivated to respond attentively, which leads to unintended issues with the quality of the data. Our question is: how do different modes of data collection - paper/pencil, computer/web-based, and smartphone - affect participants’ diligence vs. “careless responding” tendencies and, thus, the data quality? Results from prior studies suggest that different modes of data collection produce a comparable prevalence of careless responding tendencies. However, as technology develops and data are collected with increasingly diverse populations, this question needs to be readdressed and taken further by looking at cultural differences. The present research examined the effect of survey mode on careless responding across three waves in a repeated-measures design. Following recommendations in the literature, we computed a careless responding index as a composite of eight indicators that capture aspects of a participant’s inattentiveness. In a sample of working adults from China, we found that participants were significantly more careless when completing computer/web-based survey materials than in paper/pencil mode. In a sample of German students, participants were significantly more careless when completing the paper/pencil mode compared to the smartphone mode. In a sample of Chinese-speaking students, we found no difference between the modes. This paper will discuss why these results deviate from past findings that investigated study modes and hypothesize about potential cross-cultural differences.

Implementation of the European health interview survey (EHIS) into the German health update (GEDA)

BackgroundThis methodological paper describes the integration of the ‘European Health Interview Survey wave 2’ (EHIS 2) into the ‘German Health Update’ 2014/2015 (GEDA 2014/2015-EHIS).MethodsGEDA 2014/2015-EHIS is a cross-sectional health survey. A two-stage stratified cluster sampling approach was used to recruit persons aged 15 years and older with permanent residence in Germany. Two different modes of data collection were used, self-administered web questionnaire and self-administered paper questionnaire. The survey instrument implemented the EHIS 2 modules on health status, health care use, health determinants and social background variables and additional national questions. Data processing was conducted according to the quality and validation rules specified by Eurostat. ResultsIn total, 24,824 questionnaires were completed. The response rate was 27.6%. The two-stage cluster sample method seems to have been successful in achieving a sample with high representativeness. The final micro data file was inspected, approved and certified by Eurostat. Access to micro data of the EHIS 2 can be provided by Eurostat via research contract and to the GEDA 2014/2015-EHIS public use file by the Research Data Centre of the Robert Koch Institute. First EHIS 2 results are available at the Eurostat website.ConclusionsIntegrating a multinational health survey into an existing national health monitoring system was a challenge in Germany. The national survey methodology for conducting the survey had to be further developed in order to meet the overarching goal of harmonizing the health information from national statistical offices and public health research institutes across the European Union. The harmonized EHIS 2 data source will profoundly impact international public health research in the near future. The next EHIS wave 3 will be conducted around 2019.

Are Publicly Available Online Businesses Lists Appropriate to be used as Sampling Frames in Croatian Business Surveys?

Abstract Background: In order to conduct a probability business survey, a high quality sampling frame of enterprises is needed. Objectives: The goal of the paper is to investigate publicly available online businesses lists which can be used as sampling frames in business surveys in Croatia and to find out whether they have a satisfactory quality level. Methods/Approach: Publicly available businesses lists in Croatia are examined and their appropriateness for use in different modes of data collection as business sampling frames is inspected. The advantages and disadvantages of businesses lists are discussed and compared. Results: Overall 11 online businesses lists are considered as business sampling frames in the paper. The comparison analysis has shown that businesses lists from the government institutions are the best choice when business surveys are conducted face-to-face, by mail and/or by telephone. However, none of the observed businesses lists is good enough to be used in a business web survey. Conclusions: The research has shown that the publicly available online businesses lists are of a satisfactory level of quality only if traditional data collection modes are used. Unfortunately, they are not appropriate in business web surveys. Therefore, the development of a business register in Croatia is a prerequisite for conducting probability and representative business web surveys in the future.

Prevalence of cerebral palsy and intellectual disability among children identified in two U.S. National Surveys, 2011–2013

PurposeCerebral palsy (CP) and intellectual disability (ID) are developmental disabilities that result in considerable functional limitations. There are few recent and nationally representative prevalence estimates of CP and ID in the United States. MethodsWe used two U.S. nationally representative surveys, the 2011–2012 National Survey of Children's Health (NSCH) and the 2011–2013 National Health Interview Survey (NHIS), to determine the prevalence of CP and ID based on parent report among children aged 2–17 years. ResultsCP prevalence was 2.6 (95% confidence interval [CI]: 2.1–3.2) per 1000 in the NSCH and 2.9 (95% CI: 2.3–3.7) in the NHIS. ID prevalence was 12.2 (95% CI: 10.7–13.9) and 12.1 (95% CI: 10.8–13.7) in NSCH and NHIS, respectively. For both conditions, the NSCH and NHIS prevalence estimates were similar to each other for nearly all sociodemographic subgroups examined. ConclusionsDespite using different modes of data collection, the two surveys produced similar and plausible estimates of CP and ID and offer opportunities to better understand the needs and situations of children with these conditions.

Development of the European Health Interview Survey - Physical Activity Questionnaire (EHIS-PAQ) to monitor physical activity in the European Union.

BackgroundA domain-specific physical activity questionnaire (EHIS-PAQ) was developed in the framework of the second wave of the European Health Interview Survey (EHIS). This article presents the EHIS-PAQ and describes its development and evaluation processes.MethodsResearch institutes from Belgium, Estonia and Germany participated in the Improvement of the EHIS (ImpEHIS) Grant project issued by Eurostat. The instrument development process comprised a non-systematic literature review and a systematic HIS/HES database search for physical activity survey questions. The developed EHIS-PAQ proposal was reviewed by survey experts. Cognitive testing of the EHIS-PAQ was conducted in Estonia and Germany. The EHIS-PAQ was further tested in a pilot survey in Belgium, Estonia and Germany in different modes of data collection, face-to-face paper and pencil interview (PAPI) and computer assisted telephone interview (CATI).ResultsThe EHIS-PAQ is a rather pragmatic tool aiming to evaluate how far the population is physically active in specific public health relevant settings. It assesses work-related, transport-related and leisure-time physical activity in a typical week. Cognitive testing revealed that the EHIS-PAQ worked as intended. The pilot testing showed the feasibility of using the EHIS-PAQ in an international health interview survey setting in Europe. It will be implemented in all 28 European Union Member States via European Union implementing regulation in the period between 2013 and 2015. This will be a first opportunity to get comparable data on domain-specific physical activity in all 28 EU MS and to publish indicators at the EU level.ConclusionsThe EHIS-PAQ is a short, domain-specific PA questionnaire based on PA questions which have been used in large-scale health interview surveys before. It was designed by considering the respondents’ perspective in answering PA questions.Electronic supplementary materialThe online version of this article (doi:10.1186/s13690-015-0110-z) contains supplementary material, which is available to authorized users.

Evaluating a Modular Design Approach to Collecting Survey Data Using Text Messages.

This article presents analyses of data from a pilot study in Nepal that was designed to provide an initial examination of the errors and costs associated with an innovative methodology for survey data collection. We embedded a randomized experiment within a long-standing panel survey, collecting data on a small number of items with varying sensitivity from a probability sample of 450 young Nepalese adults. Survey items ranged from simple demographics to indicators of substance abuse and mental health problems. Sampled adults were randomly assigned to one of three different modes of data collection: 1) a standard one-time telephone interview, 2) a "single sitting" back-and-forth interview with an interviewer using text messaging, and 3) an interview using text messages within a modular design framework (which generally involves breaking the survey response task into distinct parts over a short period of time). Respondents in the modular group were asked to respond (via text message exchanges with an interviewer) to only one question on a given day, rather than complete the entire survey. Both bivariate and multivariate analyses demonstrate that the two text messaging modes increased the probability of disclosing sensitive information relative to the telephone mode, and that respondents in the modular design group, while responding less frequently, found the survey to be significantly easier. Further, those who responded in the modular group were not unique in terms of available covariates, suggesting that the reduced item response rates only introduced limited nonresponse bias. Future research should consider enhancing this methodology, applying it with other modes of data collection (e. g., web surveys), and continuously evaluating its effectiveness from a total survey error perspective.

Hazardous alcohol consumption among university students in Ireland: a cross-sectional study

ObjectiveThere is considerable evidence of a cultural shift towards heavier alcohol consumption among university students, especially women. The aim of this study is to investigate the prevalence and correlates of...

Contextualising brand consumption experiences: a multi-modal enabling technique

Purpose – The purpose of this paper is to elaborate on a novel multi-modal enabling technique for contextualising brand consumption experiences. Design/methodology/approach – A multi-modal interpretive narrative approach is presented as a means of investigating brands as experiential entities for use in consumer identity projects. It reports the strategic use of different modes of data collection: autobiographical narratives generated by solo participants to create a benchmark of identity and subsequent friendship pair guided discussion interviews. This offers a faster, cheaper and more convenient means of gaining access to consumer experiences of brands than traditional ethnographic methods, which require prolonged engagements within a community. Findings – Consumer narratives of actual brand consumption and of mediated brand consumption are enhanced using this method. The consumer narratives generated provided rich insights into the role of brands in contributing to national identity. The contextualised use and function of identity narratives provided by brands were identified in addition to the identification of national community rituals of consumption. Originality/value – The multi-modal use of friendship pair interviews with solo autobiographical interviews is shown to offer benefits to qualitative consumer researchers focussing on brand/identity issues. The combination of data collection methods allowed for greater reflexive, memorial and contextualised discussion in the friendship pair interviews about brand narrative consumption and generated responses that advance beyond socio-political conventions concerning brands. Consequently, contextualised brand consumption experiences can be accessed more effectively than in conventional depth interviews.

Attitude–behavior consistency, the principle of compatibility, and organ donation: A classic innovation.

The lack of consistency between peoples' attitudes toward organ donation and organ donation registration behavior has long perplexed scholars. Ajzen and Fishbein's principle of compatibility offers a potential explanation for the attitude-behavior discrepancy. This principle states that attitudes will better predict behavior if the specificity of a measured attitude matches the specificity of the behavior under consideration. Two studies, using different samples and different modes of data collection, measured general attitudes toward organ donation and specific attitudes toward registering as a donor, while simultaneously offering a registration opportunity. Compared with general attitudes about organ donation, attitudes specific to organ donor registration were superior predictors of registration intentions and behaviors. Specific attitudes explained at least 70% more variance in registration behaviors than general attitudes. The lack of attitude-behavior consistency in the organ donor domain can be partially explained by limited compliance with the principle of compatibility.

Web-Based Versus Traditional Paper Questionnaires: A Mixed-Mode Survey With a Nordic Perspective

BackgroundSurvey response rates have been declining over the past decade. The more widespread use of the Internet and Web-based technologies among potential health survey participants suggests that Web-based questionnaires may be an alternative to paper questionnaires in future epidemiological studies.ObjectiveTo compare response rates in a population of parents by using 4 different modes of data collection for a questionnaire survey of which 1 involved a nonmonetary incentive.MethodsA random sample of 3148 parents of Danish children aged 2-17 years were invited to participate in the Danish part of the NordChild 2011 survey on their children’s health and welfare. NordChild was conducted in 1984 and 1996 in collaboration with Finland, Iceland, Norway, and Sweden using mailed paper questionnaires only. In 2011, all countries used conventional paper versions only except Denmark where the parents were randomized into 4 groups: (1) 789 received a paper questionnaire only (paper), (2) 786 received the paper questionnaire and a log-in code to the Web-based questionnaire (paper/Web), (3) 787 received a log-in code to the Web-based questionnaire (Web), and (4) 786 received log-in details to the Web-based questionnaire and were given an incentive consisting of a chance to win a tablet computer (Web/tablet). In connection with the first reminder, the nonresponders in the paper, paper/Web, and Web groups were also present with the opportunity to win a tablet computer as a means of motivation. Descriptive analysis was performed using chi-square tests. Odds ratios were used to estimate differences in response rates between the 4 modes.ResultsIn 2011, 1704 of 3148 (54.13%) respondents answered the Danish questionnaire. The highest response rate was with the paper mode (n=443, 56.2%). The other groups had similar response rates: paper/Web (n=422, 53.7%), Web (n=420, 53.4%), and Web/tablet (n=419, 53.3%) modes. Compared to the paper mode, the odds for response rate in the paper/Web decreased by 9% (OR 0.91, 95% CI 0.74-1.10) and by 11% (OR 0.89, 95% CI 0.73-1.09) in the Web and Web/tablet modes. The total number of responders for NordChild declined from 10,291 of 15,339 (67.09%) in 1984 and 10,667 of 15,254 (69.93%) in 1996 to 7805 of 15,945 (48.95%) in 2011 with similar declines in all 5 Nordic countries.ConclusionsWeb-based questionnaires could replace traditional paper questionnaires with minor effects on response rates and lower costs. The increasing effect on the response rate on participants replying for a nonmonetary incentive could only be estimated within the 2 Web-based questionnaire modes before the first reminder. Alternative platforms to reach higher participation rates in population surveys should reflect the development of electronic devices and the ways in which the population primarily accesses the Internet.

Measurement invariance and quality of composite scores in a face-to-face and a web survey

Measurement equivalence is a pre-requisite to be able to make comparisons across groups. In this paper we are interested in testing measurement equivalence across respondents answering surveys done using different modes of data collection. Indeed, different modes of data collection have specific characteristics that may create measurement non-equivalence across modes. If this is so, data collected in different modes cannot be compared. This would be problematic since, in order to respond to new challenges, like costs and time pressure, more and more often researchers choose to use different modes to collect their data across time, across surveys, and across countries. Studying data about trust and attitudes towards immigration, this paper shows that measurement equivalence holds across a face-to-face and a web survey done in the Netherlands (2008-2009). Moreover, the quality estimates of the Composite Scores are quite high and pretty similar in the two surveys for the four concepts considered.